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We study two models realized by two-component Fermi gases loaded in optical lattices. We clarify 
that multi-band effects inevitably caused by the optical lattices generate a rich structure, when the 
systems crossover from the region of weakly bound molecular bosons to the region of strongly 
bound atomic bosons. Here the crossover can be controlled by attractive fermion interaction. One 
of the present models is a case with attractive fermion interaction, where an insulator-supcrfluid 
transition takes place. The transition is characterized as the transition between a band insulator and 
a Bose-Einstein condensate (BEC) superfiuid state. Differing from the conventional BCS superfiuid 
transition, this transition shows unconventional properties. In contrast to the one-particle excitation 
gap scaled by the superfiuid order parameter in the conventional BCS transition, because of the 
multi-band effects, a large gap of one-particle density of states is retained all through the transition 
although the superfiuid order grows continuously from zero. A reentrant transition with lowering 
temperature is another unconventionality. The other model is the case with coexisting attractive 
and repulsive interactions. Within a mean field treatment, we find a new insulating state, an orbital 
ordered insulator. This insulator is one candidate for the Mott insulator of molecular bosons and is 
the first example that the orbital internal degrees of freedom of molecular bosons appears explicitly. 
Besides the emergence of a new phase, a coexisting phase also appears where superfluidity and an 
orbital order coexist just by doping holes or particles. The insulating and superfiuid particles show 
differentiation in momentum space as in the high-T c cuprate superconductors. 

PACS numbers: 03.75.Ss, 05.30.Fk, 67.85.Lm, 64.70.Tg 



I. INTRODUCTION 

Control of interaction strength between particles by 
utilizing a Feshbach resonance makes it possible to form 
weakly bound molecules of two Fermi particles with con- 
trollable binding energy [2, S S, H H . Wlth ^ ^ ° f 
this controllability, crossover between the usual BCS su- 
perfiuid state to the Bose-Einstein condensation (BCS- 
BEC crossover) was observed in two-component Fermi 
gases (mixtures of two-hyperfine states) [1, 0|- In ad- 
dition to the success in tuning interactions, optical lat- 
tices formed by standing waves of light also provide us 
with ideal systems to study various phenomena, such 
as the superfluid-Mott insulator transition in Bose sys- 
tems 8]. This controllability of parameters makes ul- 
tracold atomic gases ideal model systems to investigate 
interacting many-body systems. 

Since we can control both interaction strength and 
lattice potential depth, it is possible to realize any re- 
gion of energy scales, such as E- m t ~ E g ^> £khn where 
E mt is the interaction energy, E g is the energy of band 
gaps and E^ n is the kinetic energy. Thus, we have a 
chance to study the interpolating region between solid 
state physics where E g > E- mt usually stands and molec- 
ular physics where E{ nt ^> E g stands. The intermediate 
region E- lnt ~ E g is both complicated and interesting. 
This is because the interaction mixes two bands sepa- 
rated by the band gap and therefore orbital degrees of 
freedom play important roles. In this region, treating 
molecules as a fixed minimum unit is not justified be- 



cause several different types of molecular bosons can be 
formed with different combinations of orbitals. Internal 
degrees of freedom of bosons that are often considered 
in ultracold atomic gases are the spin degrees of freedom 
of hyperfine states. It has been theoretically shown that 
because of spin degrees of freedom of bosons, there can 
be several nontrivial phases 0, [l(| El, E2, EH • However, 
roles of other types of freedom are rarely considered and 
remain open questions. In this paper, we highlight effects 
of orbital degrees of freedom arising from multi-band ef- 
fects. 

Here, we first review an intriguing experiment with 
two-component Fermi gases of 6 Li done by Zwierlein et 
al. under the condition with E m t ~ E g JlJ]. They ob- 
served the superfluidity both in the BCS and in the BEC 
sides with attractive interaction in optical lattices. We 
need to pay attention to the density of particles per unit 
cell n. In this experiment, the density of particles, n is 
equal to two, which means that there is one particle of 
each component per unit cell on average. In this optical 
lattice, a band gap between the lowest and the second- 
lowest bands is nonzero under strong periodic lattice po- 
tentials. If the gap is nonzero and there is no interac- 
tion, we should obtain a band insulator. Nevertheless, 
they observed the superfluidity for the lattice potential 
strong enough to form a band gap between the lowest 
and the second-lowest bands. In addition, they observed 
the disappearance of the superfluidity by strengthening 
the lattice potential. Then, the authors claimed that it 
was a transition between the superfluidity and the Mott 
insulator of molecular bosons. 
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Considering the experimental condition, however, we 
conclude that they observed the transition between a 
band insulator and a superfluid state. The reasons are 
the following. The added attractive interaction was 
about -Eint = 7.5 [kHz] estimated from Ref. [f|, while 
the critical band gap where the superfluidity vanished 
was about E s — 37.5 [kHz] estimated using our model 
shown in Sec. [Til The energy cost to add one fermion 
to the insulator was E g = 37. 5 [kHz] while the energy 
cost to add a pair of fermions to the insulator was 
2E g — E int = 67. 5 [kHz]. This means that low-energy ex- 
citations are fermionic rather than bosonic ones. There- 
fore, it is reasonable to conclude that the transition ob- 
served by Zwierlein et al. is the band insulator-superfluid 
transition. 



This transition occurs in the case where E- m 



E e 



stands. Since this condition is hardly realized in elec- 
tron systems and is not fully studied, unexpected mech- 
anisms or phenomena as well as internal degrees of free- 
dom of molecular bosons may play important roles. Some 
theoretical works suggested possible insulator-superfluid 
transitions caused by the attractive interactions 15l . Ilq . 
17 jQilGjl- So far, however, little is known on the charac- 
ter of the transition and on properties of phases around 
the transition point with thermal effects. Therefore, de- 
tailed analyses on the transition are desired. How the 
Mott insulator of molecular bosons emerges is also an in- 
teresting open question. 

In this paper, we first focus on a case with attractive in- 
teraction between | |)-components and | |)-components 
(we call two hyperfine states | f) and | J,)), which we 
discuss in Sec. |TTJ This condition is realized by utilizing 
a Feshbach resonance as in the experiment by Zwierlein 
et al. (l4j |. We show that the band insulator-superfluid 
transition is characterized by a remarkable feature that 
the superfluid gap is not scaled by the superfluid order 
parameter and is already large in contrast to the order 
parameter growing from zero in the vicinity of the tran- 
sition. Furthermore, the binding energy of a Cooper pair 
near the transition point is large enough that a Cooper 
pair is considered as a molecular boson. Thus, this tran- 
sition is characterized by a transition between the band 
insulator and the Bose-Einstein condensation. These fea- 
tures were mentioned in Refs. (l5l. IT^. In addition, with 
decreasing temperatures, a reentrant transition into the 
non-ordered phase appears. 

It is also illuminative to compare this emergence of 
the superfluidity with a completely different and extreme 
case of simple one-component bosonic atoms. In atomic 
Bose gases with optical lattices, the binding interactions 
which stabilize bosons, namely Bose atoms are nuclear 
or electron- nuclei Coulomb interactions, which are much 
stronger than the lattice potential. In comparison with 
the interaction scales in the first system (interacting 
fermions), the attractive interactions stabilizing atoms 
are strong enough, where the system shows the super- 
fluidity. Nevertheless, in Bose systems, there exists an 
insulator, namely the Mott insulator in optical lattices, 



if the repulsive interaction between bosons is enhanced. 

The emergence of the Mott insulator of molecular 
bosons was claimed in previous works [l6|, [T^]. They 
claimed that the Mott insulator emerges from the band 
insulator by a crossover. However, it is not a simple prob- 
lem. In a Mott state, repulsive interaction is needed to 
stabilize the insulating phase. The repulsive interaction 
between molecular bosons in attractive Fermi systems 
is caused only when molecules have a spatial extension, 
which is scaled by the s-wave scatterin g le ngth between 
opposite components of Fermi gases a s [201 ] and becomes 
negligible in the limit of Bose atoms. This means that 
the Mott insulator of molecular bosons does not exist in 
the strong coupling limit (the limit of Bose atoms) in 
attractive Fermi systems. Therefore, the Mott insulator 
is possible only when the spatial extension or finite in- 
teraction range of the molecular boson comes into play. 
We then need to introduce repulsive interaction between 
molecules caused by its spatial extension to find the Mott 
insulator. In fact, the Mott insulator of Bose atoms such 
as two-dimensional 4 He on the periodic potential of the 
substrate can be realized through van der Waals repulsive 
interaction which indeed requires the finite range inter- 
action of extended molecules. 

To realize these conditions, we introduce the second 
model, where one s-wave Feshbach resonance between 
the opposite components and two p-wave Feshbach res- 
onances one for the "| tH T)" channel and 
the other for the "| J.)" channel, are assumed. It 
is easy to realize that the repulsive interactions are mim- 
icking repulsive (van der Waals) interactions in the Bose 
atom limit. Such a system may be realized in exper- 
iments, though it is difficult so far because of large 
losses of atoms around resonance points. In addition to 
Feshbach resonances induced by magnetic fields which 
are widely used in exp eriments, optical Feshbach reso- 
nances 0, [U [2f| l26t [27 [and Feshbach resonances in- 
duced by dc electric fields 28|, [29|, [3(| HH are, in principle, 
available simultaneously. With the help of combinations 
of three different Feshbach resonances, it is possible to 
tune several interactions. 

We show in Sec. Mil that one candidate for the Mott 
insulator of molecular bosons which is associated with the 
boson Mott insulator emerges in this system in the form 
of an orbital ordered insulator (OOI) in the mean-field 
theory. The OOI is caused by the orbital internal degrees 
of freedom of molecular bosons. If we assume that all the 
molecular bosons are the same kind, these typical internal 
degrees of freedom do not exist. We also investigate the 
case where the density of particles deviates from n — 
2 and find a new phase where superfluidity and orbital 
order coexist also emerges. In this phase, the insulating 
and the superfluid particles show a sharp differentiation 
in momentum space. 

The organization of this paper is the following. In 
Sec. [Hi we focus on the case with attractive interaction. 
Setting the density of particles per site as n = 2, we clar- 
ify the origin of the band insulator-superfluid transition 
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and reveal the characteristic properties. In Sec. IIII1 we 
discuss the case where there is a repulsive interaction be- 
tween the same components of fermions coexisting with 
an attractive interaction between the same components. 
If the attractive interaction between the opposite compo- 
nent becomes strong, this system is categorized as a Bosc 
gas with repulsive interaction. In Sec. IIV[ we summarize 
the results and discuss future problems. 



II. BAND INSULATOR-BEC SUPERFLUID 
TRANSITION 

A. Model and Mean Field Approximation 

Let us consider two-component Fermi gases loaded in 
optical lattices. Both components are assumed to have 
the same mass m and the interaction between the op- 
posite components is tuned via the Feshbach resonance. 
Since a lattice potential is induced by standing waves 
of light, the 3D lattice potential with the simple cubic 
symmetry has the form 

V(r) =5i att ice-Er(sin 2 (A; L x) + sin 2 (fc L y) + sin 2 (fc L z)), 

(1) 

where is a wave number of light, E r is the recoil en- 
ergy defined as h 2 ^ 2 /2m, and the coefficient Siatticc is a 
constant which can be controlled by tuning the intensity 
of light. These conditions lead to the Hamiltonian 

H =Hj drfUr)(-^- f i + V(r))f a (r) 

a 

-U ( dr/}(r)/J(r)/i(r)/ T (r), (2) 



where /J and f a are annihilation and creation opera- 
tors of fermions, respectively, /i is the chemical potential, 
and U is the coupling constant. We first transform the 
Hamiltonian into momentum space by the Fourier trans- 
formation, such that 



U(r) 



1 



(3) 



Substituting Eq. ([3]) into the kinetic term of the Hamilto- 
nian yields 



= E(£ - n)fLf*> + E to^' ( 4 ) 

kq 



Here V q has a finite value only when one of equals to 
g, the length of the unit reciprocal vector, and the others 
vanish. By this transformation, we find the interaction 
term of the Hamiltonian 



To focus on the superfluid state, we pick up the term 
which has zero total momentum for the Cooper pairs, 
namely 



Hi n t - - q E /kt/-kl/-k'l/k' 



(6) 



kk' 



and neglect the others. Diagonalizing the one-body parts 
by the unitary transformation leads to the lattice Hamil- 
tonian 



H = 



(4 - M)Ck c k - ^ 



U 



ikglstBZ 



c kT c -ki c7 -k'i c k / T' 



(7) 



where the superscript i represents the band index and 
£ k expresses the band dispersion energy, which is a func- 
tion of the lattice potential depth lattice- Differing from 
the single-band Hubbard model, this lattice Hamiltonian 
contains more than one band because we consider not 
only the weak coupling but also the strong coupling re- 
gions, where the energy scale of the interaction strength 
is larger than that of the band gap. In this Hamiltonian, 
there is no paring interaction between particles in differ- 
ent bands because we only pick up the BCS interaction 
term Eq. ©. For mathematical convenience, we rewrite 
U/Cl as [7/f2 — ► U /N s , where iV s is the number of lattice 
sites. Though the volume f2 equals to N s Nb, where N\> 
is the number of considered bands, we set U/N^ — ► U to 
follow the conventional notation. 

To discuss the band insulator-superfluid transition 
induced by the attractive interaction, we use the ex- 
tended BCS mean field approximation introduced by 
Leggett 32j. We first introduce a superfluid order pa- 
rameter 



U 



E< 

ik 



--kl c kT/ 



(8) 



as in the conventional BCS mean field approximation. 
Using the mean field Eq. {HJ, we find a mean field BCS 
Hamiltonian 



Hmf — 



ik 



feT -hi 



-At 



"A-1 



-fcj 



N S \A\ 
U 



(9) 



To diagonalize this Hamiltonian, we use the Bogoliubov 
transformation: 



4 t =<a kt - Uk«-kl 
=-k4 =<«-ki +<* a kT' 



(10) 
(11) 



-E 

kk'q 



/k+q/2T-^-k+q/2i/-k'+q/2i/k'+q/2T- where a kcr and a k is annihilation and creation operators 



(5) 



of quasiparticles, respectively. We need to determine the 
coefficients u k and u k in order to diagonalize the mean 
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field Hamiltonian. As in the conventional BCS theory, 
we find 



where 



4" 



a + a 2 



(12) 
(13) 

(14) 
(15) 



Without loss of generality, we may set both u k and 
real. By this Bogoliubov transformation, we find diago- 
nalized Hamiltonian 



if 

kt a kT 



iVs|A| 
C/ 



(16) 




FIG. 1: Superfluid order parameter A as a function of U at 
T = 0. The solid, dotted and dash-dotted lines correspond 
to the result for the lattice potential Siatticc = 0, Siatticc = 5 
and Siatticc = 10, respectively. Quantum phase transitions to 
superfluid phase are found at nonzero U for Siattice = 5 and 

Siattice = 10. 



Numerical Results 



From this Hamiltonian, the free energy is obtained as 

ik 

-2/fc B T^ln[l + e-^/ fcBT ], (17) 

ik 

where fee represents the Boltzmann constant. The self- 
consistency condition on the mean field order parameter 
A defined in Eq. ([5]) yields the self-consistent equation or 
the gap equation 

ik ik k 

where /(e) is the Fermi distribution function. We also 
need another self-consistent equation to determine the 
chemical potential /i because in the larger U region, con- 
trary to the smaller U region, the chemical potential 
largely deviates from the Fermi energy. Using the ther- 
modynamic relation, we find 

09) 

where G = F—[iN and N is the total number of particles. 
From this relation, we find the equation 

^ =E( J - JH 1 - 2 /^))). (20) 

ik k 

which determines the chemical potential. 

With this mean field approximation, given U and 
N as input parameters, A and [i are determined self- 
consistently from Eqs. (fT5|) and (|2"0)), 



We show some of the key results of the above mean 
field approximation for the cubic lattice in the follow- 
ing. The superfluid transition induced by the attractive 
interaction has some outstanding properties that differ 
from those of the conventional BCS superfluid transition. 
Unconventional transitions of the one-particle density of 
states (DOS) and reentrant behavior should be observed 
around the transition. 

In Fig. [TJ we plot the superfluid order parameter as 
a function of the interaction strength U with n = 2 
at absolute zero temperature. While, in the absence of 
the lattice potential, the superfluid phase dominates over 
U > and the transition point is at U = 0, the insulator- 
superfluid transition occurs at a nonzero U with a finite 
band gap between the lowest and the second-lowest bands 
in the presence of the lattice potential. By assuming 
|A|/|£kl ^ 1> expansion of in the free energy yields 

^=(^-eS)i a i 2 +eBi a i 4 +°(i a i 6 )- 

ki ^Sk ki °?k 

(21) 

Here, we drop the constant term. The coefficient of the 
quartic term is definitely real and positive, while the 
quadratic term changes the sign as a function of U. This 
is a typical feature of the second-order transition in the 
Landau expansion, which is consistent with the behavior 
of the order parameter shown in Fig. [TJ This expansion is 
justified because the chemical potential is located in the 
band gap near the transition point, which means that all 
£ k are not equal to zero. 

As in Ref. [lj|, the critical value of interaction U c is 
roughly estimated as follows: Let us consider the energy 
required to excite a pair of fermions from the lower to 
the higher band. If there is no interaction between the 
particles, the energy is equal to twice the energy of the 
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band gap, namely 2E g . However, with the interaction, 
the energy is modified to about 2E g — U. This is be- 
cause the pair in the second-lowest band gains — U from 
the BCS channel which is prohibited by Pauli principle in 
the band insulator. At sufficiently large U, some configu- 
rations lower the energy than that of the band insulator. 
From this insight, we conclude that the pair formation 
in the higher bands causes this transition and that the 
transition point U c is roughly determined as U c ~ 2E„. 



1. One-Particle Density of States (DOS) 

Since both the band insulating and the superfluid 
states should show gapped DOS, it is worth investigating 
the transition of DOS over the transition. In our calcu- 
lation, DOS originating from particles in the zth-band 
D 1 (lu) is obtained by using the relation 



D'U-) = - -sgnco^ImG^fe.u;) 



(22) 



where G a (k,u>) is the Fourier component of the one- 
particle Green's function 



G ii (k,t)=-i{T t (d 1 (t)^(0))). 



(23) 



Here, T t is the time-ordering operator. Since DOS origi- 
nating from the down-spin component is the same as that 
of the up-spin component in this case, we define DOS 
D l (us) only by the Green's function of the up-spin com- 
ponent. Using the Bogoliubov transformation defined in 
Eqs. (JTUJ) and (fTTj). Green's function reads 



G i \k,t) = -iui <r t K T (*K T (o))) 

-™l 2 (T t (a\(t)al kl (0))). 
The Fourier transformation yields 



G u (k,u)) 



El + iS 



El - iS 



Substituting Eq. (25]) into Eq. ((22]), we find 



Ei)+vl 6(u + Ei) 



(24) 



(25) 



(26) 



Let us call the lowest energy band "1" with the Is sym- 
metry and the three-folded second-lowest bands "2" , "3" 
and "4" with the 2p symmetry. In Fig. [2] we show the 
one-particle DOS D^oj) and D 2 (oS) = £i= 2 ,3.4 D^uj) 
near the transition point by using Eq. (|26[) . where the 
superscript denotes the band index. In the conventional 
BCS superfluid states, the excitation gap scales with the 
superfluid order parameter and we should find the co- 
herence peak of DOS around the chemical potential. In 
Fig. [21 however, no visible change is seen around the 
chemical potential. Instead, for example, at U = 11, 
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FIG. 2: (color online) One-particle density of states (DOS) 
D(ui) at iSiattlce = 5. The dashed (blue) line shows the lowest- 
band component D 1 (lo) while the solid (red) one shows that of 
the second- lowest three bands D 2 (ui) — JT=2 3 4 ^ )l ( tJ )- The 
result at U — 10 is in the band insulating phase and those at 
U — 11, 12, 13 are in the superfluid phase. Both in the band 
insulator and in the superfluid, DOS stays always zero around 
the chemical potential uj = through the transition. 
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FIG. 3: Phase diagram in the plane of temperature T and 
interaction strength U at Siatttce = 10. 



we find a growth of the DOS D 1 ^) around io/E t ~ 1 

and that of the DOS D 2 (lu) around w/E T 2. This 

is indeed the evidence for the band insulator-superfluid 
transition. The growing DOS D 2 (lo) below the chem- 
ical potential corresponds to the bound states of the 
pair of particles in the higher bands and the growing 
DOS D l {uS) above the chemical potential corresponds to 
the bound states of the pair of holes in the lower band. 
Therefore, strongly bound Cooper pairs are formed and 
condense near the transition point. In other words, the 
insulator-superfluid transition induced by the attractive 
interaction is that between the band insulator and the 
BEC superfluid. This result also indicates that the en- 
ergy needed to excite one quasi-particle or to break up a 
Cooper pair is as large as that of the band gap E e even 
at the transition point. This means that nonzero exci- 
tation gap should exist on the transition point, which is 
consistent with previous results [HI, Il9j . Therefore, in 
the superfluid state near the transition point, the super- 
fluid order parameter A is much smaller than the su- 
perfluid gap A gap . This is in marked contrast with the 
conventional superfluid state, where the superfluid gap 
scales with the amplitude of the superfluid order param- 
eter. Photoemission spectroscopy on cold atomic gases 
recently developed [33] would reveal this unconventional 
transition of DOS and give us full understanding of the 
band insulator-BEC superfluid transition induced by the 
attractive interaction. 



2. Finite Temperature : Reentrant Transition 









2.0 




.1.5 






i—i 


1.0 


O 0.5 


i — i 






0.0 



(a) T=0.3, Slattice=0 



0.5 1 1.5 2 2.5 3 3.5 4 

A/E r 




0.5 1 1.5 2 2.5 3 3.5 4 

A/E r 




0.5 1 1.5 2 2.5 3 3.5 4 

A/E r 




0.5 1 1.5 2 2.5 3 3.5 4 

A/E r 



For experimental observations, thermal effects have to 
be clarified because it is still difficult to cool trapped 
atoms down to T ~ O.OlTp, where Tp is the Fermi 
temperature. We now discuss finite temperature ef- 
fects around the band insulator-BEC superfluid transi- 
tion point. In Fig. [31 we show the phase diagram at 
^lattice = 10 in 3D with the simple cubic symmetry ob- 
tained by the mean field approximation. An outstand- 
ing point of the phase diagram is the reentrance of the 
non-ordered phase, where the lowest critical value of 



FIG. 4: Entropy as a function of A for several choices of lat- 
tice potentials and temperatures. We take fee = 1 in this 
calculation, which means that temperature has the same unit 
as energy and entropy is dimensionless. At a nonzero lattice 
potential and at sufficiently low temperatures, the entropy has 
a peak at nonzero A, while without the potential, it mono- 
tonically decays with increase in A. 
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U c ~ 14.8 is realized at T = 0.25.E r . 

To reveal the mechanism of this reentrance, we calcu- 
late the entropy 5 as a function of the order A. The 
equilibrium state is realized at the minimum point of the 
free energy F = E — TS at finite temperatures, where 
E is the internal energy of the system. Therefore, the 
system takes the phase with the maximum entropy when 
temperature gets high. At higher temperatures, the sys- 
tem usually shows a non-ordered phase because the en- 
tropy of an ordered phase is usually smaller than that 
of a non-ordered phase. However, if the entropy of an 
ordered phase is larger than that of a non-ordered phase, 
we observe an ordered phase at higher temperatures. 

Using the thermodynamic relation 

dF 

S = -^, (27) 



02" 



we find 

ST = 



2fc B 2%[l + e~ E ^ kBT ] + 2Eif{El)] (28) 



from Eq. (TIT]) . Using this form, we plot the entropy as a 
function of A in Fig. 2] Without a lattice potential, the 
entropy takes its maximum always at A = irrespective 
of temperatures. On the other hand, with sufficiently 
strong lattice potential and at low temperatures, the en- 
tropy takes its maximum at a nonzero value of A. This 
smaller value of the entropy at A = is caused by the 
fact that the non-ordered state at A = is nearly a 
band insulator. On the other hand, when the tempera- 
ture exceeds the band gap, entropy takes its maximum 
at A = 0. Thus, at relatively low temperatures, we ob- 
serve the ordered (superfluid) phase in higher tempera- 
ture region whereas, at relatively high temperatures, we 
observe the ordered (superfluid) phase in lower temper- 
ature region. These are the reasons why we observe the 
reentrance of the non-ordered phase. 

The larger entropy in the superfluid phase is easy to 
understand when we consider detailed band structures. 
Comparing the excitation gap in Fig. [DJa) with that in 
Fig. [DJb) , we find that the gap in the superfluid phase is 
smaller than that in the band insulating phase. There- 
fore, at nonzero temperatures, thermal excitations make 
the larger entropy in the superfluid phase than that in 
the band insulating phase. 

As mentioned in Ref. [l9j , the smaller superfluid gap is 
caused by particle-hole asymmetry of our model. In fact 
in Ref. [15| , this reentrant behavior was not clear because 
the toy model of Ref. [l5| has particle-hole symmetry. 
Considering the difference between the models, we con- 
clude that larger DOS above the chemical potential than 
that below the chemical potential helps the clear emer- 
gence of the reentrance because of its large deviation from 
particle-hole symmetry. If the degeneracy of the second- 
lowest bands increases, experimental observations of the 
reentrance becomes easier. 

Although this calculation is based on the mean field ap- 
proximation, strong fluctuations, especially phase fluctu- 
ations need to be considered around the BEC transition. 



When we consider phase fluctuations, the entropy in the 
superfluid phase becomes even larger, which means that 
the reentrance becomes clearer. 

The phase diagram which we obtained suggests that 
any band insulator at T = may undergo a transition 
into superfluid or superconducting states by thermal ex- 
citations. The necessary condition is only a reasonable 
value of attractive interaction between particles though 
it is difficult to realize in other systems than the cold 
atom system. 



III. MOTT STATE OF MOLECULAR BOSONS 

In attractive Fermi systems, we found the band insu- 
lating state in weak coupling regions and the superfluid 
state in strong coupling regions as in previous section. 
We can consider the strong coupling regions of attrac- 
tive Fermi systems as Bose systems. Although, in Bose 
systems, there is a Mott insulating phase if repulsive in- 
teraction between atoms is enhanced, we could find no 
Mott state in strong coupling regions of attractive Fermi 
systems. This is because the repulsive interaction be- 
tween molecular bosons is scaled by the s-wave scattering 
length of | t)-| I) channel and goes to zero in the strong 
coupling limit. However, in Bose systems, repulsive in- 
teractions stabilizing Mott states originate from van der 
Waals interactions, which are not included in attractive 
Fermi systems. Thus, to find Mott states of molecular 
bosons, an additional repulsive interaction needs to be 
introduced. In the following, we propose a new system 
which can be realized in experiments and show one can- 
didate for the Mott insulator of molecular bosons. The 
insulator has a strong relation to orbital internal degrees 
of freedom of molecular bosons. In addition, we inves- 
tigate cases where n deviates from 2, and show a new 
coexisting phase and its typical properties. 



A. Model and Mean Field Approximation 

Since magnetic, electric and optical Feshbach reso- 
nances are available simultaneously, several types of in- 
teractions may be tuned simultaneously. In this section, 
we consider the case where | |)-| J.) channel is controlled 
by a magnetic Feshbach resonance, and | f)-| f) and 
| j,)- 1 j.) channels are controlled by optical Feshbach res- 
onances. A necessary condition to control all the inter- 
actions is that all the three resonance points are well 
separated. 

From now on, we consider the case where | j) in- 
teraction has a s-wave symmetry and is attractive with 
the value —U, and both | \)-\ f) and | J.) interactions 
have p-wave symmetries and are repulsive with the same 
value, W. Since these interactions are originally short 
ranged and gases are dilute, we can treat them as inter- 
actions within same lattice sites in lattice models. As 
an approximation, we consider the p-wave interactions 
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as homogeneous for simplicity, though they have depen- 
dence on scattering angles. Although this approximation 
may cause quantitative difference on results, there is no 
qualitative difference. 

We naively find that molecular bosons are formed be- 
cause of the attractive interaction and that molecules 
form a repulsively interacting Bose gas. However, this is 
not such a simple case as a repulsively interacting Bose 
gas. Although treating a bosonic atom as a minimum 
unit is justified, treating a molecular boson as a simple 
boson is not justified when the interaction energy be- 
tween molecules are comparable to the binding energy 
of a molecular boson. In addition, more than one en- 
ergy bands are needed to investigate the repulsively in- 
teracting molecular Bose gas because of the Pauli ex- 
clusion principle of fermions. Considering these condi- 
tions, we introduce a simplest Hamiltonian, namely a 
two-dimensional two-orbital Hamiltonian 



H ~ ^( e k ~ M) c kL- C Lr - TT ^2 C kT C ~ki C -k'i C k'T 



w 



— V r lf r 1 r 2t r 2 
jy ^ko-^kcr^kV^k'c 



(29) 



kk'o- 



where e k represents the energy dispersions 



= — 2t(cos(k x ) + cos(ky)) + At' cos(k x ) cos v 
= — 2i(cos(fc x ) + cos(ky)) + At' cos(k x ) cos(fc, 
+ AE. 



ik y ) (30) 
(31) 



Here, we retain only the Hartree-Fock and BCS terms in 
the mean field treatment for simplicity and we set the 
lattice constant unity. While the energy splitting of the 
two bands is given by the amplitude of the periodic lat- 
tice potential Siatticc in Sec. HH in this section, we here 
take the energy splitting AE as an parameter in Eq. (f3"Tj) . 
The first and the second terms in Eq. ([29]) are the same 
as in SecHH though energy dispersions are different. The 
last term in Eq. (f2"9"]l is caused by p-wave Feshbach reso- 
nances. As we mentioned above, interactions are limited 
local within same sites. Thus, interactions caused by 
p-wave resonances must be between different orbitals be- 
cause of Pauli principle. We take t = 1 as the energy unit 
and control the parameters AE and t' , which determines 
the band structure for non-interacting particles. With 
these band structures, we investigate transitions among 
superfluid, normalfluid and insulators as a function of the 
two interactions U and W. 



approximations, we introduce mean fields 

s ik 



(32) 
(33) 



s r<r 

11 =w I>r T + < - < - <y^ r ( 34 ) 



m 2 



m 3 



N, 

1 i 



y ^{n^+n^-n^-n^y^ (35) 

r 

1 J2(nl t +71^-71^ -n 2 H )e^ r , (36) 



where Q — (tt,tt) and n l ra . = (c^c^). The configurations 
of the orders mi, mi and 7713 are shown in Fig. [5] By 
using Eqs. (|3^|) - ([5d| . n\ a are written as 



'i-t 



n-\ 


- (mi + 


m 2 + 


m 3 )e i Q- r 




- (mi - 


4 

m 2 - 


m 3 )e iQr 




- (-mi 


4 

- m 2 


+ m 3 )e^ r 


nH 


- (-mi 


4 

+ mi 


- m 3 )e iQ - r 


4 



(37) 
(38) 
(39) 
(40) 



The common part n/4 is absorbed in the chemical poten- 
tial as is the Hartree term originating from the attractive 
interaction. Using the above mean fields, we find a mean 
field Hamiltonian 



2 

k+Q 



w 



+ U\A\ 2 + — (ml +m\ - m% - n 2 ). (41) 

Here, we define a 8 x 8 matrix TL 2 and a vector rf. The 
matrix TLi can be diagonalized by using generalized Bo- 
goliubov transformation, such that 



Ck =U k /3 k , 



(42) 



To solve the Hamiltonian Eq. 



by using mean field 



where Vk is a 8 x 8 matrix and the elements of a vector /3k 
are operators of quasi-particles as a k <T m Eqs. (fT0|) and 
(|11[) . Substituting this equation into the definitions of the 
mean fields yields a set of five self-consistent equations. 
In the following, as in Sec. UH we use the equation for 
the number conservation to define the chemical potential. 
Therefore, we solve these five equations self-consistently 
and obtain the orders A, mi, m 2 , m 3 and the chemical 
potential /1. In the following, we show numerical results 
of this mean field treatment. 



B. Integer-Number Filling 

In Figs. [5][51 we show phase diagrams for n = 2 ob- 
tained by the above mean field approximation. At T = 
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FIG. 5: (color online) Configurations of orders mi, mi, and 
ms. (a) is the arrangement of mi, (b) is the arrangement of 
mi, and (c) is the arrangement of 7713. Filled (blue) particles 
represent | f ) and open (red) particles are for | J.) components. 
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FIG. 6: Phase diagram in the plane of repulsive interaction 
W and attractive interaction U for AE = 0, n = 2.0, t' = 0.0 
and T — 0. The solid line is a first-ordered transition line. 
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FIG. 7: Phase diagram in the plane of repulsive interaction 
W and attractive interaction U for AE = 0, n = 2.0, t' = 0.0 
and T = 1. The solid line is a first-ordered transition line 
while the dashed lines show second-ordered transition lines. 
The open circle is a bicritical point. 
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with AE = 0, two ordered phases are seen in the phase 
diagram; one is the superfluid (SF) phase and the other 
is the orbital ordered insulating (OOI) phase. Strictly 
speaking, two types for the OOI phase exist. One is 
given by the condition with mi ^ and m,2 = m^ = 0, 
and the other is given by the condition with mi ^ and 
mi = 7713 = 0. Both types have the same energy and 
one of these degenerate two may be realized. Without 
loss of generality, we choose the type with mi 7^ as the 
OOI phase. When either T or AE differs from zero, how- 
ever, a non-ordered phase appears in the weak coupling 
region. The non-ordered phase at T — is a normalfluid 
(N) phase when \AE\ < 8 while it is a band insulating 
(BI) phase when \AE\ > 8. Phase transitions between 
the SF and the N (or BI) phase (the SF/N(BI) transition) 
are of the second order and the OOI/SF transition is of 
the first order for any value of AE. On the other hand, 
the OOI/N(BI) transition is of the second order when 
AE = and is of the first order when AE ^= 0. There- 
fore, the critical point in the phase diagram of Fig. [7] for 
AE = at T = 1 is a bicritical point. When AE is 
nonzero as in Fig. [SJ the OOI/N(BI) transition line is of 
first order. In contrast to the phase diagram of Fig. 
no multicritical point exists in this case. 



FIG. 8: Phase diagram in the plane of repulsive interaction 
W and attractive interaction U for AE = 9, n = 2.0, t' = 0.0 
and T = 0. The solid line is a first-ordered transition line 
while the dashed line shows a second-ordered transition line. 



Let us focus on the phase diagram for \AE\ > 8. In 
this case, the superfluid has two neighboring insulators, 
the band insulator and the orbital ordered insulator. As 
in the case of Sec. [Ill the BI phase cannot exist with suffi- 
ciently large attractive interaction. The OOI phase, how- 
ever, exists, which suppresses the SF phase in the larger 
U region with the repulsive interaction. Thus, if the re- 
pulsive interaction coexists with the attractive interac- 
tion, a new phase, an orbital ordered insulator emerges. 
Since this insulator is caused by the repulsive interaction 
between molecular bosons and exists even in large U re- 
gions, it is a candidate for the Mott insulator of molecular 
bosons, though there is symmetry breaking. In the OOI 
phase, there exist two types of molecular bosons: one is a 
molecular boson composed of fermions in the band-1 and 
the other composed of band-2 fermions. The emergence 
of two kinds of molecules is due to the orbital internal de- 
grees of freedom of molecular bosons, which comes from 
the fermionic degrees of freedom. Therefore, the OOI 
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phase is a typical example where the orbital internal de- 
grees of freedom of molecular bosons are observed explic- 
itly. The only example of internal degrees of freedom of 
bosons so far observed in ultracold atomic gases is the 
spin degrees of freedom of atomic bosons. In addition to 
the spin degrees of freedom of atomic bosons, the orbital 
degrees of freedom of molecular bosons may also be ob- 
served as another internal degrees of freedom of bosons. 
Although spin degrees of freedom is often discussed in 
the context of spinor BECs, the orbital internal degrees 
of freedom is so far paid less attention. Since the treat- 
ment of the spin degrees of freedom is different from that 
of the orbital degrees of freedom, more experimental and 
theoretical works are required to understand the roles of 
the orbital degrees of freedom on physical properties. 



1. Classification of Insulators 

Here, we classify insulators consisting both of fermions 
and of bosons and discuss their relations in order to com- 
pare the insulators in our phase diagrams with them. 

In Fermi systems, three types of insulators exist, one is 
a band insulator (BI), another is a Mott insulator (MI) 
and the last one is an ordered insulator (01). We as- 
sume that only the 01 has symmetry breaking. In Bose 
systems, there exist two insulators, one is a boson Mott 
insulator (BMI) and the other is a bosonic ordered insu- 
lator (BOI). 

In the superfluid side, it is established that the BCS 
superfluid and the BEC superfluid are connected each 
other by a crossover [f| BUS [HI- In the insulating side, 
however, it is complicated. What is established is that a 
transition should exist between ordered phase and non- 
ordered phase. From our phase diagrams, it is likely that 
an 01 and a BOI connect each other by a crossover. It 
is reasonable to treat the system in large U region as a 
Bose system of molecular bosons. The 001 phase in this 
region should be classified into a BOI. On the other hand, 
in small U region, the system behaves as a Fermi system. 
Therefore, the 001 phase in this region should be clas- 
sified into an 01. Thus, an 01 and a BOI are connected 
each other by a simple crossover. About two-component 
Fermi gases in optical lattices, it was claimed that a BI 
and a BMI connected each other by a crossover [l5|, [lj| • 
However, so far, it is not clearly settled. The relation 
between a BI and an MI is also a fundamental open is- 
sue. The relation between an MI and a BMI is not also 
established. In this paper, we established that, because 
of the orbital internal degrees of freedom, a BOI emerges 
in a two-component Fermi gas and that an 01 and a BOI 
are connected each other by a simple crossover. However, 
the other relations remain as future problems. 



2. Remaining Questions 

There are two points to be discussed in our phase dia- 
grams. One is that the OOI/SF transition is of the first 
order in our phase diagrams while the superfluid-Mott 
insulator transition in Bose systems is of the second or- 
der. When two local minima in the free energy entangle 
each other because of quantum fluctuations, these tran- 
sition may become continuous. In mean field theories, 
fluctuations are neglected and thus, the first order tran- 
sition is favored. By including fluctuations, however, this 
transition can be of the second order. 

The other point is the critical value of W . In our phase 
diagrams, the critical value W c is scaled as W c ~ 2U in 
large U region. In Bose systems with repulsive interac- 
tions, however, the critical value of the interaction for the 
superfluid-Mott insulator transition is typically some or- 
ders of magnitude smaller than the binding interactions 
which stabilize atoms and is not scaled as W c ~ 2/7. This 
discrepancy is caused by the overestimate of the kinetic 
energy of a Cooper pair by the mean field treatment. 
Written in real space, the BCS term we considered looks 
like a pair hopping term to arbitrary distance, namely 

_ 7y~Xl c k t T c -ki ci -k'i c k'T ~* ~Jf f drdr'cj^c^cl,^^ . 
s kk' s J 

(43) 

The kinetic energy of a Cooper pair is scaled with U 
in the mean field approximation. Therefore, it is rea- 
sonable that the critical value W c scales with U in our 
approximation. Let us consider the region where U is 
large and W = by perturbation theories. In this re- 
gion, however, the kinetic energy of a Cooper pair is es- 
timated at about t 2 /U. Thus, the critical value of the 
repulsive interaction for the OOI/SF transition should 
be scaled as W c ~ t 2 /U, which is consistent with the 
difference between W c and U by orders of magnitude in 
simple Bose systems. We believe that more sophisticated 
treatment beyond the mean field approximations would 
not face these problems, though most essential results are 
described in our mean field treatment. 

This scaling indicates that, in large U regions, the crit- 
ical value of W is relatively small. These conditions are 
realized when detuning from resonant points is larger. 
This means that such regions are favorable in experi- 
ments because losses of atoms from traps are suppressed. 

C. Noninteger Filling: Coexisting Phase 

Density of particles is also easily tuned in ultracold 
atomic gas systems. We now show phase diagrams when 
n deviates from 2 for AE = in Fig. [5] Besides the 
SF and the orbital ordered (00) phase, a new phase ap- 
pears where the SF order coexists with the orbital order 
(the OO+SF phase). Both of the 00/OO+SF and the 
OO+SF/SF transitions are of the second order in con- 
trast to the case of n = 2 where the OO/SF transition is 
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FIG. 9: Phase diagrams in the plane of repulsive interaction 
W and attractive interaction U at AE = 0, n = 1.8 and t' = 
0.0. The upper panel (a) is at T = and the lower panel (b) is 
at T — 1. In both phase diagrams, coexisting phases appear. 
The dashed lines show second-ordered transition lines. The 
open circle in the panel (b) is a tetracritical point. 
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FIG. 10: (color online) SF and OO amplitudes in momentum 
space at AE = 0, t' = 0.2, n = 1.8, W = 4 and U = 0.5. The 
upper panel is the SF amplitude while the lower panel is the 
OO amplitudes. 



of the first order. In the phase diagram at T — 1, a tetra- 
critical point is found while it is replaced with the bicrit- 
ical point when n = 2. However, at absolute zero tem- 
perature, the critical point is located at (U, W) — (0, 0) 
and the OO phase without the superfluid order vanishes. 

In the coexisting phase, the amplitude of the superflu- 
idity n s f(fe) and that of the orbital order n 00 (k) is well 
separated in momentum space. The definition of the am- 
plitudes n s f(fe) and n OQ (k) are 

n sf (k) =<cL ki 4 T ), (44) 
n QO (k) =^2( c l f +Q A, - 4+Q*cL>- ( 45 ) 

In Fig. [101 we show both the amplitude of the or- 
bital order and that of the superfluidity in momen- 
tum space. Similarly to the Fermi surface observed 
in underdoped regions of the high-T c superconducting 
cuprates (35l . l36l . |37| . the amplitude of the superfluid- 
ity is large around (tt/2,tt/2). Although this system is 
completely different from the cuprates and the similarity 
is superficial in many respects, these differentiations sug- 
gest the existence of underlying common physics [H, [39[ . 
Ultracold atomic gases are ideal systems to investigate 
how robust this differentiation in momentum space ex- 
ists when we control the interactions. 



IV. SUMMARY 

We have studied two different systems which can be 
designed by two-component Fermi gases. One is the case 
with attractive interaction between the opposite compo- 
nents, which is similar to the situation in the experiment 
by Zwierlein et al. fli| . The other is the case both with 
attractive and repulsive interactions, which is realized by 
utilizing more than one Feshbach resonances. 

In the first system, there should be a band insulator- 
BEC superfluid transition when one changes the interac- 
tion strength. This is in contrast with the claim by Zwier- 
lein et al. although they claim the transition between the 
superfluid and Mott insulator of molecular bosons. The 
origin of this transition is the energy gain coming from 
the BCS channel (or pair formation in higher bands) as 
we discussed in Sec. [TTJ This band insulator-BEC super- 
fluid transition is characterized by the following features: 
(1) The excitation gap A gap is not scaled by the super- 
fluid order parameter A in the vicinity of the transition in 
contrast to the conventional superfluid-normalfluid tran- 
sition. Indeed, at the transition point, A gap is even 
nonzero while A vanishes. This is consistent with the 
previous results in Refs. [l^, [l!|. The excitation spec- 
tra may be observed by the photoemission spectroscopy 
which is recently developed by Stewart et al. 33]. 
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(2) The transition is that between a Bose-Einstein con- 
densation of molecular bosons composed of two fermions 
and a band insulator of fermions. This is confirmed by 
the large binding energy of Cooper pairs near the transi- 
tion point. 

(3) With decreasing temperatures, a reentrant transition 
into the non-ordered phase appears. 

In the second system, in the interaction region where 
treating a molecular boson as a minimum unit is not jus- 
tified, we have found a new insulating phase, an orbital 
ordered insulator (OOI). Differing from a band insulator, 
the OOI has symmetry breaking. The symmetry break- 
ing is an outstanding example of effects of internal de- 
grees of freedom of molecular bosons which is observable. 
The OOI is also one candidate for the Mott insulator of 
molecular bosons, though it has symmetry breaking in the 
present mean-field theory and it can be replaced with a 
genuine Mott insulator without any symmetry breaking 
if more sophisticated treatment is employed for lattices 
with a geometrical frustration effect. Besides these re- 
sults, we find in general a coexisting phase in the second 



system when density of particles n deviates from 2. By 
doping holes or particles into the orbital ordered insula- 
tor, the phase where the superfluid order coexists with 
the orbital order emerges. The coexistence shows up as 
a differentiation in momentum space. 

These properties obtained in two systems are charac- 
teristic for the intermediate regions where the attractive 
interaction energy is comparable to the band gap energy 
or the repulsive interaction, where multi-band effects are 
crucial. 
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